1. Field of the Invention
The invention relates to optical systems and their elements, and to image orientation and stabilization. More particularly, the invention is directed to line of sight image stabilization by way of movable reflective structures such as mirrors. In particular, the invention concerns hyperspectral and other optical passive and active imaging systems that can benefit from three axes image stabilization as well as orientation, such as airborne and spaceborne (off-nadir viewing) imaging systems. The apparatus of the invention shows particular utility for use with airborne and spaceborne pushbroom hyperspectral and, with forward motion compensation, ultraspectral imaging systems, agricultural and military applications, or in any other application requiring three-axes line of sight image stabilization and orientation.
2. Background
The background of this invention involves imaging systems as well as image orientation and stabilization systems. Multispectral, hyperspectral, ultraspectral, and other imaging systems are well known in the art, as are image orientation and stabilization systems. Typical image stabilization systems reduce the magnitude of changes in the attitude, orientation, or location of light arriving at a detector array in a passive imaging system, or light departing the source on an active airborne system, in one or more axes. Such changes are affected by redirecting incoming or departing light or moving the detector array or source.
Certain imaging systems have particular requirement for image orientation and stabilization. For example, hyperspectral imaging systems are often of a pushbroom type (described below), benefiting from three-axis stabilization. Ultraspectral imaging systems benefit from three-axis stabilization plus forward motion control (hereinafter “FMC”) in order to allow extended un-perturbed exposure times. There is considerable prior art to address these stabilization requirements.
Common in the art are hyperspectral imaging systems (both of the pushbroom and whiskbroom type) designed to acquire an image of a narrow strip in the cross-track flight direction (typically one pixel along-track and a fixed width of N-pixels cross-track). Often, such imaging systems rely on the forward motion of the imaging system carrier (for example, an airplane, helicopter, satellite, unmanned aerial vehicle (UAV), or other such equipment) to “sweep” forward. A subsequent narrow strip image is acquired after a fixed amount of time for forward motion. This process can repeat for any length of time. The multiple narrow strip images are “stitched” together to form a complete image, of the fixed N-pixel width, and of a length, equal to the number of snapshots taken while in forward motion. Such an imaging system is typically referred to as a pushbroom imaging system. Pushbroom and other imaging systems are well known in the art. Such system can benefit from image orientation and stabilization as each subsequent image preferentially must align closely with its predecessor image. When the carrier platform (hereinafter “CP”) executes attitude changes due to wind gusts or air turbulence, the orientation of the narrow image strip changes and the scene may be under or over sampled. The orientation and stabilization system should maintain each strip in the cross track direction and centered on the nadir view of the platform. Without image stabilization, the series of images are nearly impossible to stitch to form a single, contiguous image of the scene.
Approximately twenty years ago, in one of the earliest pushbroom system patents, Minott (U.S. Pat. No. 4,407,563) acknowledged this issue. Minott explained that the pitch, roll, and yaw of a satellite had to be known in order to compensate for these movements in the post-image processing. While Minott dealt with aberrations due to the use of beam splitters, the problem of compensating for movement in three dimensions has remained critical for multispectral imaging.
In the intervening years, many techniques have been developed for improving image stabilization. These techniques fall into several general categories. The first are the entirely gimbal-mounted systems. A gimbal is simply a device with two equally perpendicular and intersecting axes of rotation, giving free angular movement in two directions. When an imaging apparatus is mounted in a gimbal, the imaging system can target any object in a very large field of regard (hereinafter “FOR”), often encompassing half a sphere or more. Moreover, such a system can remain pointed at a single target and can then compensate for attitude changes in three axes: pitch, roll, and yaw. Gimbals are often seen in police and search and rescue helicopter mounted flood lamps and television-style cameras that can point at, and track, any object on the ground.
A disadvantage of the gimbal systems, however, is that while they can compensate for pitch, roll, and yaw movements, unimportant to tracking systems, television cameras, and flood lamps, the gimbal cannot maintain orientation. Orientation changes occur either when the carrier (typically for tracking setups, a helicopter) approaches or recedes from a target in an off-nadir trajectory, or when the carrier has a speed relative to the tracked object. For example, the orientation of an image captured by a camera mounted in a gimbal underneath a helicopter changes as the helicopter pursues a ground vehicle and catches up to the ground vehicle (in order to fly above the target so that ground units may visually locate the target by observing the more visible helicopter). For a typical gimbal-mounted television camera, this orientation change is unimportant: the television viewer simply sees the target from various angles. The important feature is that the target is somewhat centered in the frame, but the direction that the target is moving relative to the “bottom of the screen” is irrelevant.
In a pushbroom imaging system or in an automated target identification system (comparing the target image to a known database of targets), however, these orientation changes result in columns of the scanned ground grid not being aligned or the identification failing for lack of a steady target image. In the pushbroom case, instead of obtaining nice columns of image, intensive post-imaging processing is needed to compensate for the orientation changes, and often, at best, only the center of the image will be useful, though likely no part of the image will be useful. Where the CP performs a nose pitch upward, certain strips of image will be missing completely and the lost data will not be recoverable. In the target identification case, if the target cannot remain steady in the image, comparison software will not have a sufficiently detailed image from which to make database comparisons (for example, identifying a vehicle, person, or other target of interest). In Ultraspectral Imaging systems, the orientation problem makes registering a series of interferograms impracticable.
A further disadvantage of gimbal systems is that the instrumentation used for imaging must fit within a gimbal. That means that special equipment is needed for use with a gimbal, or at the least, the gimbal may need to be extremely large to accommodate a large imaging apparatus. Larger equipment also requires larger motors, which in turn increases the total bulkiness, cost, and weight of such systems.
A second technique to provide image stabilization is the use of a scanning mirror, located between the imaging system optics (the lens) and the detector array. Such a scanning mirror can be very small and compact. By pivoting the scanning mirror, the image can be redirected to the detector array quickly and effectively. This provides an obvious advantage to cost and system size. However, there are several disadvantages to such systems. The first is that a single scanning mirror can only compensate for two axes of movement. The scanning mirror, like the gimbal system (and in fact, the mirror can be mounted in a very small gimbal within the camera optics), can compensate for pitch and roll, but not yaw. Yaw, or a change in orientation of the CP, cannot be changed with a single mirror. A further disadvantage of this art is that such a scanning mirror must be built into the optics, which then requires special arrangement of the fore optics of the sensor. A scanning mirror of this type cannot be added to an existing optical system without significant retrofit cost and technology. This is a costly consideration for existing imaging systems.
Finally, the third, and heretofore most accurate of the typical image stabilization techniques is the use of a large “flatbed” on which to mount the optics system. The flatbed is motorized to move in all directions, allowing the imaging system to be tilted to compensate for pitch, roll, and yaw. In fact, by way of a series of complex calculations, appropriate pitch, roll, and yaw compensations can combine to further provide FMC, as well. Although such flatbed image stabilizations systems appear to resolve the orientation and stabilization problem for pushbroom scanners, there are significant disadvantages to these systems. Such imaging system stabilization control must move the entire imaging system. In order to stabilize images for large optical systems, the motors and entire flatbed system must be very robust. Such motors need to be able to quickly move a large mass, accurately, and in constantly changing directions. Such motors are very expensive, thus making flatbed stabilization not very cost effective. If less expensive motors are used, the flatbed stabilization cannot respond to changes of direction as quickly and the image is less (or perhaps insufficiently) stabilized.
Most recently, several companies have began testing new stabilization systems, including combining a pivoting, gimbal-type system with a Dove Prism (NovaSol, in Hawaii, US) and combining two mirrors in the imaging system optics and further allowing the entire optics assembly to move in order to provide three axes of stabilization (Zeiss Optronik GmbH, Germany, U.S. Pat. No. 6,370,329). The gimbal-based system continues to suffer from the limitations imposed by requiring the optical system to fit within a gimbal, while the mirrored, moveable optics requires special imaging systems. These systems thus still do not provide for an economically feasible method of retrofitting image stabilization to existing systems, or provide compact, cost-effect image stabilization to new systems.
Accordingly, there is a need in the art for a simple, effective, and cost efficient image orientation and stabilization system that overcomes or avoids the above problems and limitations.